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INTRODUCTION 

The  number  of  young  people  eligible  for  conscription  into  the  German  Federal  Armed  Forces  is  decreasing, 
making  the  successful  planning  and  execution  of  manpower  management  plans  and  policies  increasingly 
important.  In  conjunction  with  the  decreasing  number  of  potential  conscripts,  the  necessity  for  savings  requires 
the  downsizing  of  the  Psychological  Service  of  the  Federal  Armed  Forces.  Thus,  the  various  tasks  performed  in 
processing  personnel  for  entrance  into  the  armed  forces,  including  screening,  assessment,  and  placement,  must  be 
performed  by  fewer  people.  Accomplishing  this  workload  with  fewer  personnel  resources  can  be  achieved,  in 
part,  by  leveraging  the  effectiveness  of  the  remaining  personnel.  Time  spent  on  routine  tasks  must  be  reduced  to 
allow  the  remaining  psychologists  to  focus  on  the  situations  and  decisions  that  require  their  individual  attention 
and  expertise. 

All  military  offices  staffed  by  the  Psychological  Service  of  the  Bundeswehr  using  psychological  diagnostics  will 
be  equipped  with  computer-assisted  testing  (CAT)  facilities.  These  CAT  facilities  will  use  computers  to  perform 
many  of  the  tasks  that,  historically,  have  been  accomplished  by  professional  personnel.  This  computer-based 
support  is  a  key  to  leveraging  the  productivity  of  the  reduced  psychological  staff.  As  part  of  this  effort  most 
effectively  to  leverage  the  expertise  of  the  remaining  psychologists,  a  CAT-assisted  expert  system  wilt  be 
designed,  developed,  evaluated,  deployed,  and  implemented.  TTiis  system  will  provide  a  knowledge  base 
containing  expertise  of  the  psychologists  and  a  set  of  rules  for  making  decisions  based  upon  their  expert 
knowledge.  The  plan  is  to  incorporate  the  new  expert  system  into  the  "Computer-Assisted  and  Adaptive  Testing 
(CAT)  of  the  Psychological  Service,"  an  approved  system.  2 

The  development  of  the  proposed  expert  system  is  being  accomplished  in  two  phases:  (1)  a  market  survey  of  the 
existing  products  and  conceptual  models,  and  (2)  development  of  the  actual  system,  specifically  designed  for  the 
Psychological  Services  working  environment.  The  work  described  in  this  paper  addresses  the  first  phase. 

The  study  requirement  entailed  a  review  of  the  existing  products  and  research  literature  on  expert  systems,  as 
applied  to  personnel  management.  Early  in  the  study,  it  was  determined  that  focusing  solely  on  expert  systems 
would  exclude  a  vast  amount  of  information  that  was  relevant  to  the  objectives  of  the  study.  Many 
computer-based  systems  and  models  provide  information  that  personnel  decision-makers  find  very  useful.  These 
systems  and  models,  while  not  strictly  speaking  "expert  systems,"  are  computer-based  Decision  Support  Systems 
(DSS).  Therefore,  it  was  decided  to  expand  the  scope  of  the  study  and  address  the  broader  subject  of  DSSs  for 
personnel  management.  This  would  include,  but  not  be  limited  to,  expert  systems. 


1This  paper  is  based  upon  a  completed  project,  fully  documented  in  the  following  report:  Sands,  W.  A.  (1996). 
Planning  an  Expert  System  for  Use  by  the  Psychological  Service  of  the  German  Federal  Armed  Forces  in 
Determining  the  Suitability  of  Conscripts  for  Basic  Military  Service.  CRA  Report  96-0 1 .  Arlington,  VA: 
Chesapeake  Research  Applications. 

Performance  Specification  for  the  Study:  "Expert  System  for  the  Support  of  Psychological  Diagnostics  for  the 
Suitability  Examination  and  Suitability  Determination  of  Conscripts  for  Basic  Military  Service  (Short  title:  EUF 
Expert  System)." 

Other  approaches  which  appear  to  offer  promise  include  neural  networks  and  genetic  algorithms.  In  addition, 
many  of  the  models  in  use  by  the  various  U.S.  Armed  Services  address  personnel  flows,  projections,  resource 
allocations,  and  person-job  matching  algorithms.  All  of  these  approaches  fall  under  the  DSS  umbrella.  The 
products  and  the  literature  reviewed  for  this  study,  as  well  as  the  interviews  conducted,  reflect  this  broader  scope. 
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APPROACH 

The  first  step  in  reviewing  products  was  to  identify  candidate  products.  This  was  accomplished  by  reviewing  the 
research  literature  and  advertisements  in  professional  journals  describing  commercially  available  products  that 
seemed  promising.  A  facsimile  (fax)  letter  was  sent  to  each  of  the  companies  identified,  explaining  the  purpose 
of  the  study  and  requesting  product  information.  Regular  mail  was  used  for  those  companies  having  no  listed  fax 
numbers. 

Interviews  were  scheduled  and  conducted  with  sixteen  individuals  having  a  variety  of  types  and  levels  of 
responsibilities  for  personnel  management  decision  support  systems.  The  responsibility  levels  of  the  interviewees 
ranged  from  manpower  analysts  to  policy-makers. 

A  computer-based  literature  search  was  conducted  in  two  very  large  reference  databases:  (I)  the  Educational 
Resources  Information  Center  (ERIC),  and  (2)  the  on-line  version  of  Psychological  Abstracts  (PsycINFo). 

RESULTS  AND  DISCUSSION 


Products 

Product  information  was  requested  from  forty-five  companies.  Information  was  received  from  twenty-nine 
companies.  The  information  obtained  from  four  of  these  companies  indicated  that  the  products  were  not  relevant 
to  the  objectives  of  the  study.  Product  reviews  were  conducted  on  the  information  received  from  the  remaining 
twenty- five  companies.  The  quantity  and  quality  of  the  information  provided  by  the  companies  covered  a  wide 
range,  from  a  few  pages  of  information  to  extensive  packages  containing  product  brochures,  feature  descriptions, 
product  reviews,  examples  of  product  applications,  information  on  the  company,  training  opportunities, 
product-ordering  information,  and  demonstration  diskettes.  A  total  of  284  printed  items  was  received,  containing 
over  800  pages  of  information.  In  addition,  five  of  the  companies  provided  diskettes  containing  product 
information  and/or  product  demonstrations. 

Products  were  reviewed  for  eight  criteria  specified  in  the  project  requirements:  (1)  open/relational  database 
structure,  (2)  psychological  diagnostic  rules,  (3)  psychological  diagnostic  classification,  (4)  robust  for  incomplete 
datasets,  (5)  case-based  reasoning  (CBR)  techniques,  (6)  complex  statistical  links,  (7)  complex  filters  or  rule 
algorithms,  and  (8)  recommendation  for  use.  Companies  were  rated  "Yes"  on  an  individual  criterion  if  the 
materials  they  submitted  indicated  that  their  product  would  support  that  criterion,  or  if  support  for  the  criterion 
could  be  readily  inferred  from  the  information  received.  Companies  were  rated  "No"  if  their  materials  explicitly 
indicated  a  lack  support  for  that  criterion,  or  if  no  information  supporting  a  '"Yes"  evaluation  could  be  readily 
inferred  from  the  materials  received.  Eight  companies  were  recommended  for  consideration  as  primary  tools,  six 
companies  were  recommended  as  secondary  tools,  and  the  remaining  eleven  companies  were  not  recommended 
for  further  consideration. 

Interviews 

A  total  of  sixteen  people  were  interviewed,  including  manpower  managers  with  policy-making  and  operational 
responsibilities,  and  researchers  with  experience  in  the  design  and  development  of  decision  support  systems.  The 
information  presented  below  summarizes  various  comments  and  suggestions  obtained  during  the  interviews. 
Many  of  the  points  were  raised  independently  by  multiple  people.  The  comments  reflect  the  different 
perspectives  contributed  by  policy-makers,  operational  analysts,  and  researchers. 

Decision  Support  Systems  offer  many  important  advantages.  They  enable  analysts  to  address  complex  questions 
that  arise  from  within  their  own  organizational  hierarchy  or  from  outside  (e.g.,  Congressional  staffers)  in  a  timely 
manner.  The  information  is  quantitative,  accurate,  consistent,  verifiable,  and  defensible. 

The  models  leverage  the  limited  resources  (personnel  and  time)  which  can  be  dedicated  to  addressing  a  problem. 
Answers  that  once  took  days  to  produce  can  be  generated  in  hours  or  even  minutes.  Alternatively,  more  scenarios 
can  be  addressed  in  a  fixed  period  of  time. 

Various  problems  associated  with  some  DSS  models  were  also  identified.  For  example,  the  development  time  for 
a  model  sometimes  becomes  so  long  that  the  problem  which  initiated  the  development  effort  has  disappeared  by 
the  time  the  model  is  available,  in  other  cases,  the  model  turns  out  to  be  prohibitively  expensive. 

Some  models  are  so  complex  and  cumbersome  to  use  that  they  "die  under  their  own  weight."  This  can  happen 
when  researchers  become  focused  on  using  the  "latest  and  greatest"  methodological  approaches  and  lose  sight  of 
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the  original  rationale  for  the  model.  While  this  scientific  focus  may  lead  to  many  journal  article  publications  and, 
thereby,  enhance  the  professional  stature  of  the  researcher,  it  does  not  contribute  to  solving  the  original  problem. 
If  a  model  is  either  too  complex  to  be  understood  or  too  cumbersome  to  use  (or  both),  it  will  not  be  fully  utilized. 

Personnel  turnover  presents  other  potential  problems..  For  example,  transfer  of  knowledge  about  a  model  to 
incoming  personnel  may  not  be  adequate  (and  may  be  nonexistent).  If  the  model  documentation  is  poor,  the 
likelihood  of  full  model  utilization  diminishes.  Another  potential  problem  is  that  new  personnel  may  adopt  a  "not 
invented  here"  attitude  about  a  model  if  they  played  no  role  in  its  development.  The  high  turnover  in  many  units 
makes  these  problems  particularly  important  considerations  in  military  organizations. 

Finally,  if  a  model  is  delivered  to  the  user  without  adequate  testing,  serious  problems  can  result.  If  the  model 
developer  shortcuts  the  test  and  evaluation  phase,  some  problems  may  not  be  discovered  and  solved  prior  to 
operational  use.  When  a  model  suddenly  produces  results  that  are  obviously  erroneous  to  an  experienced 
decision-maker,  the  credibility  of  the  model  can  be  seriously  damaged.  The  problem  might  be  very  easy  for  the 
developer  to  fix,  but  the  model’s  (and  perhaps  the  modeler’s)  reputation  can  remain  tarnished.  In  most  cases, 
extensive  and  thorough  testing  can  prevent  this  type  of  problem. 

A  number  of  useful  recommendations  were  offered  by  the  persons  interviewed.  Identification  of  the  key  players 
in  a  model  development  effort  is  essential.  The  model  developer  should  work  very  closely  with  the  customer(s) 
before,  during,  and  after  the  model  is  built.  Many  times,  this  "handholding"  is  the  key  to  sustained  model  use. 

The  customer  must  define  the  model  objectives  as  precisely  as  possible.  Key  operational  constraints  should  be 
identified. 

An  extreme  concern  for  global  optimization  in  a  DSS  can  be  counterproductive.  The  forced  translation  of 
quantities  that  are  naturally  expressed  in  different  metrics  into  a  common  metric  (to  provide  an  index  of 
optimization)  can  reduce  a  model’s  usefulness. 

An  alternative  approach  is  to  supply  the  decision-maker  with  the  tradeoff  information  on  the  components  and  let 
the  person  integrate  the  information  and  make  a  decision.  This  suggestion  was  described  as  "keeping  the  human 
in  the  decision  loop." 

Models  should  be  built  and  tested  incrementally.  Evaluation  of  model  results  should  emphasize  producing 
information  that  leads  to  the  correct  decision,  not  statistical  accuracy.  Users  should  be  prepared  for,  and 
comfortable  with,  approximate  answers.  It  is  possible  for  a  model  answer  to  be  only  moderately  accurate  from  a 
statistical  standpoint,  yet  be  very  valuable  if  it  influences  the  policy-maker  to  make  the  best  decision,  especially 
in  large-scale  or  high-cost  applications.  Models  developed  for  different  purposes  should  be  used  to  check  on  each 
other  whenever  possible.  Finally,  a  concern  for  "user  friendliness"  in  the  model  interface  should  be  paramount  to 
facilitate  user  acceptance  and  support. 

In  summary,  the  vast  majority  of  persons  interviewed  strongly  supported  the  value  of  DSSs.  They  stated  that 
these  systems  provide  critical  information  to  the  decision-makers  in  a  timely  fashion.  In  large  organizations  (e.g., 
the  military),  even  small  improvements  can  yield  considerable  benefits.  The  people  interviewed  believed  that  the 
funds  invested  in  DSSs  yield  a  high  return  on  investment. 

Literature  Search 

The  initial  step  in  the  search  strategy  was  to  determine  the  keywords  that  would  identify  those  references  relevant 
to  the  study  objectives  ("hits").  In  some  cases,  the  search  for  a  single  keyword  yielded  an  excessive  number  of 
references.  When  this  situation  occurred,  an  additional  keyword  was  paired  with  the  initial  keyword  to  narrow 
and  refine  the  search.  For  example,  the  keyword  "evaluation"  produced  over  1  00,000  hits  in  the  ERIC  database, 
because  such  a  broad  search  locates  references  involving  "evaluation"  that  are  totally  unrelated  to  the  purpose  of 
this  study.  Narrowing  the  search  by  pairing  "artificial  intelligence"  with  "evaluation"  produced  a  total  of  139  hits. 
The  number  of  hits  for  a  keyword  was  different  for  each  database.  Therefore,  the  paired  keyword  searches 
required  were  somewhat  different. 

After  determining  the  number  of  hits  in  a  database  for  a  particular  keyword,  the  abstract  of  each  targeted 
reference  was  reviewed.  Relevant  references  were  included  in  the  computerized  reference  database  produced  as 
part  of  this  study.  Having  the  reference  information  in  a  computerized  database  enables  the  user  to  sort  the 
references  with  flexibility,  according  to  different  needs. 

RECOMMENDATIONS 
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No  existing  commercial  product  or  DSS  developed  by  a  military  organization  was  discovered  that  could  be  used 
directly,  without  further  development,  to  meet  the  needs  of  the  Psychological  Service  for  an  expert  system  for 
use  with  potential  conscripts  for  the  German  Federal  Armed  Forces.  It  is  necessary  to  build  such  a  system,  either 
by  developing  entirely  new  software  (not  recommended)  or  by  using  a  commercially-available,  off-the-shelf, 
expert  system  shell  and  custom-tailoring  a  system  for  the  operational  environment. 

Eight  of  the  twenty-five  commercial  products  reviewed  could  serve  as  a  vehicle  for  developing  a  custom  DSS. 
Information  from  these  eight  companies  should  be  evaluated  in  greater  depth,  using  the  results  of  the  ongoing 
needs  assessment  study  and  a  more  detailed  and  comprehensive  list  of  technical  and  operational  criteria.  Input 
for  developing  the  new  evaluation  criteria  should  be  provided  by  both  policy  and  operational  personnel. 
Examples  of  technical  criteria  might  include  ease  of  programming  the  system,  computer  platforms  supported, 
computer  hardware  requirements  (e.g.,  random  access  memory  size,  hard  disk  storage  capacity),  flexibility  to 
accommodate  future  changes,  etc.  Examples  of  operational  criteria  might  include  ease  of  operation  and  costs 
(purchase  base  price,  license  fees,  etc). 

The  most  promising  DSS  models  identified  in  the  technical  literature  should  be  reviewed  further.  It  is  possible 
that  some  of  the  operational  problems  identified  in  the  field  (but  not  the  entire  system)  could  be  addressed  using 
these  models.  Additional  information  that  would  be  useful  to  obtain  on  the  most  promising  existent  DSS  models 
might  include  availability,  documentation,  programming  language,  etc. 

It  would  be  beneficial  for  the  Psychological  Service  of  the  Bundeswehr  to  convene  a  meeting  of  experts  in  the 
application  of  expert  systems  and  DSS  models  to  military  personnel  management  problems.  This  meeting  could 
provide  valuable  insights  for  planning,  building,  and  implementing  the  required  system  for  use  by  the 
Psychological  Service. 
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